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In re Application of: HULLENDER et al. 
Serial No.: 09/528,889 



REMARKS 



Introduction 

The Examiner has rejected claims 1, 2, 7, 8 t 12, 20, 21 and 23 under 35 U.S.C. § 
103(a) over U.S. Patent No. 5,881,172 to Pintsov (hereinafter Pintsov). Claims 19 and 22 
were rejected under 35 U.S.C. 103(a) as being unpatentable over Pintsov in view of 
Shimuzu et al., U.S. Patent No. 6,038,343 (hereinafter Shimuzu). Claims 13-15, 17, 24 
and 25 were rejected under 35 U.S.C. 103(a) as being unpatentable over Pintsov in view 
of Crane et al., U.S. Patent No. 4,531,231 (hereinafter Crane). Claims 16, 18, 26 and 27, 
as well as claims 3-6 and 9-11 were rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pintsov in view of Crane, and further in view of Guo et al. 
"Classification trees with neural network feature extraction," Proceedings IEEE Computer 
Society Conference on Computer Vision and Pattern Recognition, June 1992 (hereinafter 
Guo). Additionally, claims 1-27 were rejected under 35 U.S.C 112, first paragraph. 
Applicants respectfully traverse these rejections. For the reasons discussed in detail 
below, all of the pending claims are in condition for allowance. 

Prior Art Rejections 

/. Pintsov fo/Ts to disclose or suggest claim limitations 

In the following, applicants provide an overview of their invention and of Pintsov 
and then discusses their differences. Applicants' technique relates to recognizing 
chirographs in a computer system by using a primary classifier and secondary classifiers 
to yield a high probability in character recognition, A primary recognizer is provided for 
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converting chirographs to code points or shape indexes, and secondary recognizers are 
developed and trained to differentiate chirographs which produce selected code points. 
The output of training data from a primary recognizer may be used to construct secondary 
recognizers for each code point supported by the system. Using the training data, the 
primary recognizer returns a code point to a sorting process to sort the chirographs and 
actual code points into various files. Note that if other types of shape indexes are being 
used, the chirographs are similarly sorted into files for each shape index based on the 
shape index returned by the primary recognizer A secondary recognizer may then be 
constructed for each of the files so that a secondary recognizer exists for each code point 
(or shape index) supported by the system. If the secondary recognizer has only one 
possible output that is the same code point as output by the primary recognizer, or 
actually reduces recognition accuracy with respect to the primary recognizer's output, 
such a secondary recognizer may be discarded in applicants' technique since it does not 
improve recognition of the chirograph. Such a sorting process automatically discovers 
any chirographs that the primary recognizer tends to confuse and secondary recognizers 
constructed for each of these files are then inherently able to better distinguish any 
chirographs that the primary recognizer tends to confuse. 

Once the recognition mechanism including the secondary recognizers are 
developed, the recognition mechanism may be used to recognize a chirograph. The 
recognition mechanism in applicants' technique may submit the chirograph to the primary 
recognizer and receives a code point (or shape index) from the primary recognizer. If a 
code point, then the code point may then be used to determine if the code point has a 
secondary recognizer associated with it. If not, the primary recognizer's returned code 



9 



Received from < 425 838 8957 > at 9/1 1/03 7:27:07 PM [Eastern Daylight Time). 




Sep 11 03 04:3Gp 



Michal ik 



(425) 83G-8957 



p, 14 



In re Application of: HULLENDER et al. 
Serial No.: 09/528,889 

point is returned by the recognition mechanism as the returned code point. However, if a 
secondary recognizer is associated with the code point received from the primary 
recognizer, the chirograph is submitted to the secondary recognizer. The code point 
returned by the selected secondary recognizer is returned as the returned code point. 
Furthermore, a plurality of secondary recognizers can be associated with a single 
character in applicants' technique. For example, a first CART tree can be provided as a 
secondary process for differentiating two-stroke M A"-shaped characters, and a second, 
distinct CART tree for differentiating three-or-more-stroke tt A"-shaped characters. 

Although CART trees have been attempted for handwriting recognition and have 
been rejected because they are unable to make reliable decisions from among large 
numbers of characters, the use of CART trees as a secondary recognition mechanism in 
applicants' technique has been thoroughly tested, and for certain confusion pairs 
advantageously has a 99.7 percent accuracy rate. The 0.3 percent error rate is believed 
to result from characters too poorly written even for humans to discern, and in fact is 
comparable to the recognition error rate of humans. 

Pintsov relates to a technique for hierarchical character recognition. Pintsov 
describes a hierarchy of specialist classifiers, each configured to recognize characters 
belonging to distinct ambiguity classes. A suspicious character may belong to a distinct 
ambiguity class based on apparent size, the type of character, level of gray in the image 
of the character, styles of handwriting, prior knowledge that a character candidate has 
been surgically separated from an adjoining character, or upon assignment of a character 
candidate by the universal classifier to a predefined character group known to be 
ambiguous. Ptinsov first applies a universal classifier system to input image data and 
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identifies "suspicious" characters. A suspicious character may be identified by the 
universal classifier system making an assignment of the character to a predefined 
character group known to be ambiguous. For a suspicious character identified, the input 
image data is then applied to a specialist classifier that is designed to handle a well- 
defined set of recognition cases. The specialist classifiers are not designed to process 
characters of other shapes. A classifier of this kind is trained only on a large set of 
characters that belong to a specific ambiguity class. 

In Pintsovs technique, image data is applied to a universal classifier system which 
can comprise one or more universal classifiers known in the art. For unambiguous data, 
the universal classifier system outputs a character code. If the universal classifier system 
determines that a candidate character for the input data is "suspicious", then the universal 
classifier system selects a "specialist* classifier based upon the probable identity of the 
candidate character. The specialist classifier applies a recognition algorithm tailored to 
the candidate character and outputs a probable character code. The specialist classifier 
may use feature extraction algorithms such as neural networks and syntactic or linguistic 
algorithms, nearest neighbor algorithms and other algorithms known in the art of character 
recognition. 

Significantly, Pintsov does not describe a primary recognizer able to provide a 
shape index without making any decision as to whether that chirograph is of a set of easily 
confused chirographs, as generally recited in applicants' claims. Nor does Pintsov 
describe creating a secondary classifier for each code point output by training data from a 
primary recognizer, as in applicants' technique. Rather, Pintsov describes specialist 
classifiers configured to recognize characters belonging to predefined ambiguity classes 
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and a universal classifier system that first determines that a character candidate does not 
belong to a predefined ambiguity class before outputting the character candidate. As 
described, a group of characters belonging to a predefined ambiguity class are used to 
identify whether a candidate character is a suspicious character. Each specialist 
classifier is trained or built to distinguish only differences between characters belonging to 
distinct ambiguity classes. A suspicious character may belong to a distinct ambiguity 
class based on apparent size, the type of character, level of gray in the image of the 
character, styles of handwriting, or upon assignment of a character candidate by the 
universal classifier to a predefined character group known to be ambiguous. 

In Pintsov*s technique, a universal classifier first outputs a probable identity of a 
candidate character and the universal classifier system then makes a determination as to 
whether the character is suspicious. In one example described in Pintsov, such a 
determination may be made by checking whether the candidate character belongs to a 
predefined character group known to be ambiguous. In such a case, Pintsov describes 
that a character may be pre-programmed to belong to a predefined character group 
known to be ambiguous when it is known to be mistaken for another character within that 
same ambiguity class, (e.g. "4" and a g"). Thus to determine whether a candidate 
character is suspicious, the candidate character may be compared to any pre- 
programmed characters belonging to a predefined character group known to be 
ambiguous. However, nowhere in Pintsov is there a description of a universal classifier 
outputting a character without making any decision as to whether that chirograph is of a 
set of easily confused chirographs. 
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The following table lists the elements of claim 1 and the prior art sections upon 



which the Examiner has relied as allegedly disclosing those elements. 





Claim 1 Element 


Prior Art 


A. 


providing a primary recognizer 
for converting chirographs to 
shape indexes, the primary 
recognizer providing output 
including a shape index when a 
chirograph is input thereto 


... The invention first applies a universal classifier 
system (which may comprise one or more universal 
classifiers) to input image data, and identifies 
"suspicious" characters .... 

(Pintsov, 2:7-34, 2:28-54,2:21,2:34-35) 


B. 


providing a plurality of 
secondary recognizers to 
convert chirographs into code 
points, and associating the 
secondary recognizers with at 
least some of the shape 
indexes 


... the invention employs a hierarchy of "specialist 
classifiers, each configured to recognize characters 
belonging to distinct ambiguity classes. ... 

...If the character is suspicious, a specialist 
classifier for the suspicious character is selected* ... 

(Pintsov, 3: 16-18, 4:10-14, 3:18-21) 


C. 


receiving a chirograph 


... automated recognition of machine-printed or 
handwritten characters. ... 
(Pintsov, 1:10-11) 


D. 


providing the chirograph to the 
primary recognizer and 
receiving a shape index 
therefrom, the primary 
recognizer providing the shape 
index without making any 
decision as to whether that 
chirograph is of a set of easily 
confused chirographs 


... or upon assignment of a character candidate by 
the universal classifier system to a predefined 
character groups known to be ambiguous (e.g., the 
pair tt 4" and "9"...). 

The universal classifier system can be pre- 
programmed , for example, to always recognize that 
a "4" is a suspicious character because .... 
(Pintsov, 3:55-60, 4:4-10) 


E. 


determining whether one of the 
secondary recognizers is 
associated with the shape 
index, and if so, selecting that 
secondary recognizer as a 
selected secondary recognizer 
and passing the chirograph to 
the selected secondary 
recognizer, the secondary 
recognizer returning a code 
point 


One or more universal classifiers are applied to 
image data to generate a probable character (step 
100). ... 

(Pintsov, 3:65-4:22) 
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Element D of claim 1 recites "providing the chirograph to the primary recognizer 
and receiving a shape index therefrom, the primary recognizer providing the shape index 
without making any decision as to whether that chirograph is of a set of easily confused 
chirographs." The sections of Pintov allegedly disclosing this element do not describe a 
primary recognizer providing a shape index without making any decision as to whether 
that chirograph is of a set of easily confused chirographs. Rather, Pintsov describes 
specialist classifiers configured to recognize characters belonging to predefined ambiguity 
classes and a universal classifier system that first determines that a character candidate 
does not belong to a predefined ambiguity class before outputting the character 
candidate. As described in Pintov, a group of characters belonging to a predefined 
ambiguity class are used to identify whether a candidate character is a suspicious 
character. Each specialist classifier is trained or built to distinguish only differences 
between characters belonging to distinct ambiguity classes. (See Col 3:15-25.) A 
suspicious character may belong to a distinct ambiguity class based on apparent size, the 
type of character, level of gray in the image of the character, styles of handwriting, or upon 
assignment of a character candidate by the universal classifier to a predefined character 
group known to be ambiguous. (See Col 3:50-60.) 

In Pintsov's technique, a universal classifier first outputs a probable identity of a 
candidate character and a determination is then made as to whether the character is 
suspicious. (See Cols 3:47-50; 3:66-4:2.) In one example described in Pintsov, such a 
determination may be made by checking whether the candidate character belongs to a 
predefined character group known to be ambiguous. (See Col 3:57-60.) In such a case, 
Pintsov describes that a character may be pre-programmed to belong to a predefined 

14 



tecelved from < 425 836 8957 > at 9/1 1/03 7:27:07 PM [Eastern Daylight Time]. 





Sep 11 03 04:37p 



M i cha 1 i k 



(425) 836-8957 



p. 19 



In re Application of: HULLENDER et al. 
Serial No.: 09/528,889 

character group known to be ambiguous when it is known to be mistaken for another 
character within that same ambiguity class, (e.g. "4" and "9"). (See Cols 4:2-1 0; 3:57-60.) 
Thus to determine whether a candidate character is suspicious, the candidate character 
may be compared to any pre-programmed characters belonging to a predefined character 
group known to be ambiguous. However, nowhere in Pintsov is there a description of a 
universal classifier outputting a character without making any decision as to whether that 
chirograph is of a set of easily confused chirographs. Pintsov explicitly states that "a 
determination is then made as to whether the character is "suspicious" and then provides 
an example where candidate character "4" may be determined to be a suspicious 
character by finding it pre-programmed as a character belonging to the predefined 
character group or ambiguity class containing "4" and "9". 

The particular sections of Pintsov allegedly disclosing this element describe instead 
determining a suspicious character based upon assignment of a character candidate by 
the universal classifier system to a predefined character group known to be ambiguous 
(e.g., the pair "4" and "9"; the group "I", T, "1", etc.). In the context that Pintsov is 
explaining that the "universal classifier system can be pre-programmed to always 
recognize that a '4' is a suspicious character because it is often mistaken for a '9"\ 
Pintsov is describing explicitly assigning the character "4" to the ambiguity class 
containing u 4 n and U 9 M . Pintsov goes on to confirm this by concluding "[t]hus, the 
candidate character is part of the ambiguity class containing '4' and '9"\ 

Applicants respectfully submit that dependent claims 2-6, 19, 20 and claims 7-18, 
21-27, by similar analysis, are not anticipated by Pintsov. Each of these claims include 
the limitations of a "without making any decision as to whether that chirograph is of a set 
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of easily confused chirographs." As discussed above regarding claim 1, the sections of 
Pintov allegedly disclosing this element do not describe a primary recognizer providing a 
shape index without making any decision as to whether that chirograph is of a set of easily 
confused chirographs. Rather, Pintsov describes specialist classifiers configured to 
recognize characters belonging to predefined ambiguity classes and a universal classifier 
system that first determines that a character candidate does not belong to a predefined 
ambiguity class before outputting the character candidate. 

//. The combination of Pintsov with Crane. Shimuzu and/or Guo is impermissible by 



For a combination of prior art references to render an invention obvious, there must 
be some reason, suggestion, or motivation found in the prior art whereby a person of 
ordinary skill in the field of the invention would make the combination. In re Oetiker, 977 
F.2d 1443, 1447, 24 USPQ2d 1443, 1446 (Fed. Cir. 1992). A finding of obviousness on 
any other basis would constitute impermissible hindsight. See Interconnect Planning Corp. 
v. Feil, 774 F.2d 1132, 1143, 227 USPQ 543, 551 (Fed. Cir. 1985). Otherwise, combining 
prior art references without evidence of such a suggestion, teaching, or motivation simply 
takes the inventor's disclosure as a blueprint for piecing together the prior art to defeat 
patentability— the essence of impermissible hindsight In re Dembiczak, 175 F.3d 994, 
999, 50 USPQ2d 1614, 1617 (Fed. Cir. 1999). 

In the present application, the Office action has essentially done what is not proper 
by law, and used applicants' teachings as a blueprint, using an (incorrectly) modified 
Pintsov for some of the claimed limitations, and, without any specific evidence of 



law 



16 



Received rrom<42S 836 8957 > at 9/11/03 7:27:07 PM [Eastern Daylight Time}. 





Sep 11 03 04:38p 



M i cha 1 i k 



(425) 836-8957 



p. 21 



In re Application of: HULLENDER et al. 
Serial No.: 097528,889 

motivation to combine, has hunted for other references that might supply the limitations 
present in the application but missing from Pintsov. Instead of presenting any specific 
evidence of motivation to combine, the Office action has only made conclusory statements 
that are wholly unrelated to the claims in order to allege obviousness. However, such 
broad conclusory statements regarding the teaching of multiple references, standing 
alone, are not evidence of obviousness. Id. 

For example, in rejecting claims 19 and 22, the Office action refers to a single, final 
character recognition block (FIG. 1, element 17) that works with distance values and 
feature vectors, to illogicafly allege that this is a per-index unique secondary recognizer as 
claimed. In support, the Office action contends that it would be obvious to "use the writer 
specific feature vectors of Shimuzu to augment the system of Pintsov to increase the ratio 
of a character recognition system employing a universal recognition dictionary without 
requiring that special operations be performed before character recognition is performed 
on the handwriting of a new writer." Not only is such an allegation merely a broad, 
conclusory statement not found in the prior art of record or elsewhere, but even if true, it is 
wholly unrelated to the claimed subject matter of claims 19 and 22, which is essentially 
directed towards having a unique secondary recognizer for each shape index that may be 
output by the primary recognizer. In other words, at best Shimuzu appears to be able to 
recognize things differently per writer, not per shape index. For at least the foregoing 
reasons, claims 19 and 22 are clearly patentable over any permissible combination of 
Pintsov and/or Shimuzu. Reconsideration and withdrawal of the rejections of claims 19 
and 22 is respectfully requested. 
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Moreover, regarding the combination of Pintsov with Guo r applicants note that Guo 

teaches away from any such combination, by teaching that its modified classification tree 

is a whole recognizer unto itself that provides many benefits over CART trees and neural 

networks, if anything, this would essentially lead an objective reader to conclude that 

there is no need for secondary recognition. In the specific example of handwriting 

recognition, Guo explicitly points out (at page 186, section 4.2) that "it is a 10 class 

problem to recognize the numbers 0 - 9," which is like the typical recognition attempts 

referred to in applicants* background section. Moreover, Guo separately tests his 

"CTNNFE" recognition method on samples against a neural network recognition method 

(see Guo, Fig. 4.2) and determines that his single classification tree method is superior in 

many ways to other methods, again leading the reader away from the present invention. 

Applicants reiterate that when prior art, in any material respect teaches away from the 

claimed invention, the art cannot be used to support an obviousness rejection. In re 

Geisler, 116 F.3d 1465, 1471, 43 USPQ2d 1362, 1366 (Fed Cir. 1997). Applicants 

respectfully request withdrawal of the §1 03(a) rejections of the claims based in any way 

on Guo. 

Rejections Under 35 U-S.C. 112 

The Examiner has rejected claims 1-27 under 35 U.S.C 112, first paragraph, as 
containing subject matter which was not described in the specification in such a way as to 
reasonably convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. Applicants respectfully 
traverse these rejections. Applicants respectfully submit that claims 1, 7, 13 and 18 
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properly recite the limitation that the primary recognizer provides the shape index "without 
making any decision as to whether that chirograph is of a set of easily confused 
chirographs.* 

The Examiner misinterprets the sections of the specification allegedly describing 
that the primary recognizer makes decisions as to whether a chirograph is of a set of 
easily confused chirographs. Those decisions are not made by the primary recognizer at 
all, but to the extent they can be considered decisions, are made during the training 
process. During each actual recognition of a chirograph, the decision as to whether to 
use a secondary recognizer is not made by the primary recognizer, but is inherent by the 
existence or non-existence of a secondary recognizer for a code point The specification 
states, "In accordance with one aspect of the present invention, as described in more 
detail below, those chirographs which often confuse a recognizer are provided to a 
secondary recognition process." As described above in the overview of applicants' 
technique, a primary recognizer is provided for converting chirographs to code points, and 
secondary recognizers are developed and trained to differentiate chirographs which 
produce selected code points. The output of training data from a primary recognizer may 
be used to construct secondary recognizers for each code point supported by the system. 
Using the training data, the primary recognizer returns a code point to a sorting process to 
sort the chirographs and actual code points into various files. Note that if other types of 
shape indexes are being used, the chirographs are similarly sorted into files for each 
shape index based on the shape index returned by the primary recognizer. A secondary 
recognizer may then be constructed for each of the files so that a secondary recognizer 
exists for each code point (or shape index) supported by the system. If the secondary 
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recognizer has only one possible output that is the same code point as output by the 
primary recognizer, or does not improve recognition accuracy, such a secondary 
recognizer may be discarded in applicants' technique since it does not improve 
recognition of the chirograph. Such a sorting process, as part of training, automatically 
discovers any chirographs that the primary recognizer tends to confuse whereby 
secondary recognizers constructed for each of these files are then inherently able to 
distinguish any chirographs that the primary recognizer has tended to confuse in training. 
Following training, when used to recognize an actual unknown chirograph, those 
chirographs that have characteristics similar to those training samples that tended to 
confuse the primary recognizer are thus distinguished by the secondary recognizers, 
without any decision made by the primary recognizer as to whether that chirograph is of a 
set of easily confused chirographs. This is well explained in the specification; any inability 
to understand applicants' invention is because of misinterpretations made by the Office 
action in the (failed) attempt to fit applicants' invention into Pintsov's very different 
teachings. Applicants respectfully submit that dependent claims 2-6, 8-12, 14-17 and 19- 
27, by similar analysis, are in condition for allowance. 
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Conclusion 

Based upon the above remarks, all of the pending claims are in condition for 
allowance. Applicant respectfully requests reconsideration of this application and its early 
allowance. If in the opinion of the Examiner a telephone conference would expedite the 
prosecution of the subject application, the Examiner is invited to call the undersigned 



attorney at (425) 836-3030. 

Respectfully submitted, 




Albert S. Michalik, Reg. No. 37,395 

Attorney for Applicants 

Law Offices of Albert S. Michalik, PLLC 

704 - 228th Avenue NE, Suite 193 

Sammamish, WA 98074 

(425) 836-3030 

(425) 836-8957 (facsimile) 

OFFICIAL 
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